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DETAILED ACTION 

Allowable Subject Matter 

1 . Claims 16-22, 52-56, 75-80 are objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any Intervening claims. 

Claim Rejections - 35 USC § 102 



2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 



3. Claims 1-15, 43-51, 57-60, 67-74, and 81-85 are rejected under 35 U.S.C. 102(e) 
as being anticipated by Cossatto et al., (6,1 12,177). 

As per claim 1 , Cossatto et al., teach a method of enhancing audio renderings of 
non-audio data sources, comprising: 

detecting a nuance of a non-audio data source (Fig.2, item 18); 
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locating an audio cue corresponding to the detected nuance (Fig.2, item 19); 

and, 

associating the located audio cue with the detected nuance for playback to a 
listener (Fig.2, items 14, 15). 

As per claim 2, Cossatto et al., teach the method according to claim 1, further 
comprising creating an audio rendering of an non-audio segment of the non-audio data 
source, wherein the non-audio segment is associated with the nuance, and, mixing the 
associated audio cue with the audio rendering of the segment (Fig.2, items 14 and 15). 

As per claim 3, Cossatto et al., teach the method according to claim 1, wherein 
detecting a nuance of a non-audio data source detects a plurality of nuances of the non- 
audio data source, locating an audio cue locates audio cues for each of the detected 
nuances, and associating the located audio cue with the detected nuance for playback 
to a listener associates each of the located audio cues with the respective detected 
nuance, and further comprising creating an audio rendering of the non-audio data 
source, and mixing the associated audio cues in with the audio rendering (Fig. 3b). 

As per claim 4, Cossatto et al., teach the method according to claim 3, wherein 
mixing the associated audio cues occurs while playing the audio rendering to the 
listener (Col .4, lines 3-41). 

As per claim 5, Cossatto et al., teach the method according to claim 3, wherein 
the non-audio data source is a text file and wherein creating an audio rendering of the 
non-audio data source further comprises processing the text file with a text-to-speech 
translator (Col.4, lines 3-41). 
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As per claim 6, Cossatto et al., teach the method according to claim 3, wherein at 
least one of the detected nuances is presence of a formatting tag (Col.8, lines 25-39). 

As per claim 7, Cossatto et al., teach the method according to claim 3, wherein 
the non-audio data source is a text file and at least one of the detected nuances is a 
change in color of text in the text file (Col.9, lines 18-31, Col. 10, lines 21-26). 

As per claim 8, Cossatto et al., teach the method according to claim 1 , wherein 
the non-audio data source and the detected nuance is a change in font of text in the text 
file (Col.9, lines 33-40, Col.10, lines 11-26). 

As per claim 9, Cossatto et al., teach the method according to claim 1 , wherein 
the non-audio data source is a text file and the detected nuance is presence of a 
keyword for the text file (Col.9, lines 1-15, Col.8, lines 55-67). 

As per claim 10, Cossatto et al., teach the method according to claim 9, wherein 
the keyword is supplied by a creator of the text file (Col.9, lines 1-15, Col.8, lines 55-67). 

As per claim 1 1 , Cossatto et al., teach the method according to claim 9, wherein 
the keyword is programmatically detected by evaluating text in the text file (Col.9, lines 
1-15, Col.8, lines 55-67). 

As per claim 12, Cossatto et al., teach the method according to claim 3, wherein 
the non-audio data source is a text file and at least one of the detected nuances is 
presence of an emoticon in the text data (Col.6, lines 43-57, Col.5, lines 19-31). 

As per claim 13, Cossatto et al., teach the method according to claim 1, wherein 
the detected nuance is a change of topic in the non-audio data source (Col.5, lines 20- 
38). 
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As per claim 14, Cossatto et a!;, teach the method according to claim 6, wherein 
the formatting tag is a new paragraph tag (Col. 10, lines 16-26). 

As per claim 15, Cossatto et al., teach the method according to claim 3, wherein 
at least one of the detected nuances is a degree of certainty in translation of the non- 
audio data source from another format (Col. 10, lines 16-26). 

As per claim 43, Cossatto et al., teach a system for enhancing audio renderings 
of non-audio data sources, comprising: 

means for detecting a nuance of a non-audio data source (Fig.2, item 18); 

means for locating an audio cue corresponding to the detected nuance (Fig.2, 
item 19); and, 

means for associating the located audio cue with the detected nuance for 
playback to a listener (Fig.2, items 14 and 15). 

As per claim 44, Cossatto et al., teach the system according to claim 43, further 
comprising means for creating an audio rendering of an non-audio segment of the non- 
audio data source, wherein the non-audio segment is associated with the nuance, and, 
means for mixing the associated audio cues in with the audio rendering while playing 
the audio rendering to the listener (Fig.2, items 14 and 15, Col.4, lines 3-41). 

As per claim 45, Cossatto et al., teach the system according to claim 44, wherein 
the non-audio data source is a text file and wherein creating an audio rendering of the 
non-audio data source further comprises processing the text file with a text-to-speech 
translator (Col.4, lines 3-41). 
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As per claim 46, Cossatto et al., teach the system according to claim 43, wherein 
at least one of the detected nuances is presence of a formatting tag (Col.8, lines 25-39). 

As per claim 47, Cossatto et a!., teach the system according to claim 43, wherein 
the non-audio data source is a text file and at least one of the detected nuances is a 
change in font of text in the text file (Col.9, lines 18-31, Col. 10, lines 21-26). 

As per claim 48, Cossatto et al., teach the system according to claim 43, wherein 
the non-audio data source is a text file and at least one of the detected nuances is 
presence of an emoticon in the text file (Col.6, lines 43-57, Col.5, lines 19-31 

As per claim 49, Cossatto et al., teach the system according to claim 43, wherein 
the detected nuance is a change of topic in the non-audio data source (Col.5, lines 20- 
38). 

As per claim 50, Cossatto et al., teach the system according to claim 46, wherein 
the formatting tag is a new paragraph tag (Col. 10, lines 16-26). 

As per claim 51 , Cossatto et al., teach the system according to claim 43, wherein 
at least one of the detected nuances is a degree of certainty in translation of the non- 
audio data source from another format (Col. 10, lines 16-26). 

As per claim 57, Cossatto et al., teach the system according to claim 43, wherein 
the non-audio data source is an email message and at least one of the detected 
nuances is an email convention found in the email message (Col. 10, lines 16-26). 

As per claim 58, Cossatto et al., teach the system according to claim 43, wherein 
the non-audio data source is text provided by a user (Col. 10, lines 16-26). 
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As per claim 59, Cossatto et a!., teacli the system according to claim 43, wherein 
the detected nuance is embedded within the non-audio file (Col. 10, lines 16-26). 

As per claim 60, Cossatto et al., teach the system according to claim 43, wherein 
the detected nuance comprises metadata associated with the non-audio file (Col. 10, 
lines 16-26). 

As per claim 67, Cossatto et al., teach a computer program product for 
enhancing audio renderings of non-audio data sources, the computer program product 
embodied on one or more computer-readable media and comprising: 

computer readable program code that is configured to detect one or more 
nuances of a non-audio data source (Fig.2, item 18); 

computer readable program code that is configured to locate an audio cue 
corresponding to each of the detected nuances (Fig.2, item 19); and, 

computer readable program code that is configured to associate the located 
audio cues with their respective detected nuances for playback to a listener (Fig.2, 
items 14 and 15). 

As per claim 68, Cossatto et al., teach the computer program product according 
to claim 67, further comprising computer readable program code that is configured to 
create an audio rendering of a non-audio segment of the non-audio data source, 
wherein the non-audio segment is associated with the nuance, and, computer readable 
program code that is configured to mix the associated audio cue with the audio 
rendering of the segment (Fig.2, items 14 and 15, Col .4, lines 3-41). 
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As per claim 69, Cossatto et a!., teach the computer program product according 
tot claim 68, wherein the non-audio data source is a text file, and wherein the computer 
readable program code that is configured to create further comprises computer 
readable program code that is configured to process the text file with a text-to-speech 
translator (Col.4, lines 3-41). 

As per claim 70, Cossatto et a!., teach the computer program product according 
to claim 67, wherein the non-audio data source is a text file and at least one of the 
detected nuances is a change in color of text in the text file (Col.9, lines 18-31, Col. 10, 
lines 21-26). 

As per claim 71 , Cossatto et al., teach the computer program product according 
to claim 67, wherein the non-audio data source is a text file and the detected nuance is 
presence of a keyword for the text file (Col.9, lines 1-15, Col.8, lines 55-67). 

As per claim 72, Cossatto et al., teach the computer program product according 
to claim 71 , wherein the keyword is supplied by a creator of the text file (Col.9, lines 1 - 
15, Col.8, lines 55-67). 

As per claim 73, Cossatto et al., teach the computer program product according 
to claim 71, wherein the keyword is programmafically detected by evaluating text in the 
text file (Col.9, lines 1-15, Col.8, lines 55-67). 

As per claim 74, Cossatto et al., teach the computer program product according 
to claim 67, wherein at least one of the detected nuances is a degree of certainty in 
translation of the non-audio data source from another format (Col. 10, lines 16-26). 
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As per claim 81 , Cossatto et a!., teacli the computer program product according 
to claim 67, wherein at least one of the detected nuances is an identification of a creator 
of the non-audio data source (Col. 10, lines 16-26). 

As per claim 82, Cossatto et al., teach the computer program product according 
to claim 81 , wherein the identification is used to locate stored preferences of the creator 
(Col.10, lines 16-26). 

As per claim 83, Cossatto et al., teach the computer program product according 
to claim 67, wherein the non-audio data source is an e-mail message (Col.10, lines 16- 
26). 

As per claim 84, Cossatto et al., teach the computer program product according 
to claim 67, wherein the detected nuance is embedded within the non-audio file (Col.10, 
lines 16-26). 

As per claim 85, Cossatto et al., teach the computer program product according 
to claim 67, wherein the detected nuance comprises metadata associated with the non- 
audio file (Col.10, lines 16-26). 

Conclusion 

4. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Dutta et al., (6,453,294) teach dynamic destination determined multimedia avatars for 
interactive on-line communications. 
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Stentiford et al., ( 5,384,701 ) teach a language translation system. 
Hyde-Thomson et al., (6,487,533) teach unified messaging system with automatic 
language identification for text-to-speech conversion. 

L'Esperance et al., (US 2002/0055844) teach a speech user interface for portable 
personal devices. 

Siegel (6,442,523) teaches a method of auditory navigation of text. 
Shepard et al., (US 2003/0191682) teach a positioning system for perception 
management. 

Butler et al., (5,844,158) teach voice processing system and method. 
Freeland et al., (US 2003/0028380) teach a system for generating an audio message 
over a communications network that is at least partly in a voice representative of a 
character generally recognizable to a user. 

Goldberg et al., (6,125,175) teach a method and apparatus for inserting background 
sound in a telephone call. 

Ball et al., (6,459,774) teach structured voicemail messages. 

Johnson et al., (5,434,910) teach a method and system for providing multimedia 

substitution in messaging systems. 

Bogard (6,757,365) teaches instant messaging via telephone interfaces. 

Jayaratne (US 2003/01 15059) teaches real time translator and method of performing 

real time translation of a plurality of spoken languages. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Vijay B. Chawan whose telephone number is (571) 272- 
7601 . The examiner can normally be reached on Monday Through Friday 6:30-3:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 



supervisor, Richemond Dorvil can be reached on (571) 272-7602. The fax phone 
number for the organization where this application or proceeding is assigned is (571) 
273-8300. 

Infornnation regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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